| IndyBob: | Test |
|---|---|
| smarx: | Hi. |
| pstatho: | so can we start with question? |
| smarx: | Go for it! |
| argodev: | I'm wondering where we can learn more about health monitoring... i.e. if a worker process ties up the CPU for a long time, will the fabric think it is hosed and restart? |
| argodev: | or, more specifically, how to tell the health monitoring platform that we expect it to tie up the CPU for x amount of time and not to worry about it until after that period |
| Bill___: | Should we keep asking questions or wait until the other are answered? |
| smarx: | We're informal here. :) Sorry, multitasking... |
| smarx: | Argodev, there was some recent discussion about this... (what if you don't reply fast enough to a health query)... |
| Bill___: | What si our story on Alerting? From the service and from applications? |
| pstatho: | so what's the deal with the new projects for billing? Is there an easy way to transfer data that is in one storage account to another? |
| smarx: | @argodev I actually don't know the answer, but I can try to find that discussion and get back to you. (Drop me an email?) |
| argodev: | thanks... will do. |
| smarx: | Bill, the alerting story is changing to a pluggable model, so we're moving away from the Live Alerts dependency. |
| Bill___: | But to what? |
| smarx: | I don't have a lot more alerting details for you now, but there will basically be a way to log things and separately consume those logs and decide what to do with them. |
| smarx: | (Sorry for a bit of vagueness.) |
| Gaurav: | What about better query support in Table Storage? Is this something in the pipeline? When can we expect that? |
| smarx: | @pstatho Are you part of the billing CTP? |
| smarx: | @pstatho Or are you asking about what happens when we hit commercial launch in November? |
| pstatho: | billing ctp |
| IndyBob: | Steve, what happens if the server that is running locks-up from too much traffic (or would it) and a roll-up hasn't been programmed... |
| pstatho: | I want to use it, but all my data is in my PDC08 project |
| argodev: | any chance we could get alternate formats (i.e. JSON) from azure storage? at least for some items? (lists/metadata from blobs, table data, queue data) |
| smarx: | I actually don't know how the billing CTP works, but there isn't a way to transfer data from one account to another. |
| smarx: | @IndyBob We don't do auto-scaling if that's what you're asking. You'll have to go to the portal and add nodes (or use the API in the future). |
| IndyBob: | Would the first server reboot on it's own |
| smarx: | @argodev I'd really like us to (JSON in particular), but I haven't seen this on the roadmap yet. (I'll recommend it again.) |
| smarx: | @IndyBob It will be restarted if it crashes, but I believe the more common case is that the IIS queue fills up and you start refusing new connections. |
| pstatho: | +1 for JSON |
| Gaurav: | "use the API in the future" will these be available by November when Azure goes commercial? |
| smarx: | Yeah, did anyone see my tweet a while ago that basically said I hate XML and JSON is always better? :) |
| smarx: | @Gaurav, Yup, service management APIs are coming before commercial launch. |
| pstatho: | Atom is even more verbose |
| argodev: | yes, I did... just hoping that it would turn into more of a concrete implementation :D |
| smarx: | They'll let you deploy, scale, update with a web API. |
| pstatho: | Velocity was planned to be released about this time frame, will this be added to Azure for commercial launch? |
| argodev: | that's a good question... any chance there will be a CTP or early acess to the api? |
| smarx: | @pstatho Totally. There are some nice things Atom includes, but it's really heavy. |
| Gaurav: | Cool thanks ... and I am assuming there would be similar APIs for billing as well. |
| smarx: | @pstatho I don't actually know, but I believe the right primatives are in place (inter-role-instance communication) to run something like Velocity. (Just not sure if the integration work will be done.) |
| smarx: | @argodev I'm not sure exactly when it will be ready, so not sure if it comes in CTP before launch or just right at launch. |
| smarx: | @Gaurav No, there won't be an API for billing information. |
| smarx: | Hey, is anyone entering the Twilio contest? |
| smarx: | Did you guys see my awesome app? :) wazsong.cloudapp.net |
| Gaurav: | By billing, I meant some APIs to track the usage. Recently I signed up for something which told me that I will receive emails about the usage on a daily basis. So far, no emails of that sort. |
| pstatho: | One of the pains/frustration I have with developing with Azure, is not realling know what is coming down the pipe and when...given that Azure stuff is pretty much bleeding edge, is there any chance the Azure team can be more open with its roadmap? |
| argodev: | given a scenario wherein you want to do directX file manipulation in your worker role, I'm assuming one would have to include the directx files in the project as they are likely not enabled/installed on the worker nodes... correct? |
| smarx: | @Gaurav Maybe that's the billing CTP. I'm not actually part of that. There won't be APIs to retrieve usage, though. |
| smarx: | @pstatho We'll talk a bit more about roadmap at PDC... happy to talk about other things before then (but forgive me if I'm a bit vague). |
| argodev: | I agree with @Gaurav... the API needs to expose some sort of usage data so we can do internal costing |
| smarx: | @argodev I would think so. |
| pstatho: | so are we getting secondary indexes before commercial launch? |
| pstatho: | ;) |
| Gaurav: | rather than getting a surprise bill at the end of the month :) |
| smarx: | @pstatho I believe we're NOT getting those at commercial launch. |
| smarx: | There will be a way to get that usage (before the bill at the end of the month), just not a programmatic way. |
| argodev: | can that be added to the feature request list? |
| argodev: | has there been any conversation about lower-level access, or inter-node communication for workers? (think MPI-ish style stuff) |
| smarx: | @argodev Definitely passing on the feedback that a billing API would be useful. |
| smarx: | Communication between role instances is coming along. I haven't seen the final design, but I believe there will be an API that gets you IP addresses, and then you can communicate directly (TCP-based). |
| smarx: | So there's a layer of indirection introduced, but otherwise existing strategies for communication should work. |
| argodev: | do you know if it will be protocol restricted? or open? or node-specific firewalls? (lots of issues there...) |
| smarx: | @argodev You have to declare up front what ports you want to use. Then it's unrestricted. |
| pstatho: | sounds cool |
| IndyBob: | Is there a primary cost before usage expense or a minimum monthly fee before usage calculation? |
| argodev: | interesting... wondering how I keep @brentcodemonkey or anyone else from talking to my worker roles and limiting it just to my nodes |
| azurejournal: | Any details on payment services in the works similar to Amazon Requester pay/devPay ? |
| smarx: | You're still in your own virtual network. So it's unrestricted only within your application. |
| smarx: | @azurejournal We won't have something like "requester pay" at PDC. I think it's an interesting model though, not sure how much it's been talked about. |
| mh415: | I've been away from Azure for about 6 months. Is Dynamic Data Center Toolkit the same as on-premise Azure? http://www.microsoft.com/virtualization/cloud-computing/privatecloud.mspx |
| smarx: | We're working with PayPal to make sure their payment platform works well with Windows Azure. |
| azurejournal: | it is an easy way to sell your data ... if you have data to sell :) |
| smarx: | @mh415 I don't actually know what that is. :) |
| smarx: | @azurejournal Totally. |
| pstatho: | are there any plans to make deployments faster by giving us an option to just deploy a package instead of having to spin up new VMs each time? |
| argodev: | @mh415 brings up a good question... I know you all get this alot, but any news on a private #azure platform? |
| smarx: | @pstatho Yeah, there's some work around that. Wait for PDC. :) |
| pstatho: | why is everyone so interested in private azure? Is it because of the data sensitivity? /just curious |
| pstatho: | good to hear |
| kmarshall: | i agree on pstatho's queston. Comparing deployment on app engine to azure, azure deployment is brutally slow. App engine takes about 10 seconds |
| smarx: | Private Azure... that's a complicated question that I'll cleverly not answer. :) |
| argodev: | @pstatho - some data we work with is, well, restricted. Also, consider the notion of a first order "local" cloud that could then "burst" into the public cloud as needed for scale |
| mh415: | I'm interested in private clouds because at some point it becomes more economical to run my own datacenter again. I haven't investigated Azure with respect to this, but I believe with S3 the threshold is around 2TB. If you have less than that, S3 is more economical. More than 2TB, and you're better off hosting it yourself. |
| smarx: | Yeah, GAE isn't spinning up VMs... our model is very VM based, but there are things we can do to speed up deployment and especially updates. |
| IndyBob: | Any update on Q: 15:24 |
| argodev: | the forced programming model of azure is really nice (separation of roles, etc.) and it would be nice to tell your corporate/scientific devs to do *all* their apps in that model |
| argodev: | allowing for a smooth move to public cloud services as necessary |
| argodev: | @smarx... thanks... I think... that was the answer I was expecting but not hoping for. |
| smarx: | @mh415 The economics are definitely interesting. Mostly what we're saying is: Use Windows Server, System Center, etc. on-premises. Use Windows Azure in the cloud. |
| smarx: | Those two do need to get a bit closer together, particularly for people who want the option of deploying in _either_ place (instead of deciding up front). |
| mh415: | @smarx: yep. It'd be great to have a seamless transition between on-premise and the cloud though. e.g. I'd like to use the same Azure Tables API for both scenarios. |
| smarx: | @argodev Yeah, I do like the idea of adopting the same programming model in both places. |
| smarx: | @mh415 Yup, we're in agreement. (I just don't like to call it "private Azure" :-P) |
| argodev: | any conversations/likely hood that we'll see HPC Server 2008 available as an azure option? |
| smarx: | So no one's entering the Twilio contest? :-) |
| azurejournal: | how would you call it @smarx? |
| azurejournal: | :) |
| mh415: | I can't tell you how happy I am with Azure Tables. In a past life, I tried forcing MySQL to behave like ATS. My life is much easier now. |
| smarx: | @argodev We're at the level of "What would make sense?" on the HPC and Windows Azure front. I don't know what will come out of that. |
| smarx: | @mh415 It's great to hear that. A lot of people get frustrated by the restrictions. |
| smarx: | @mh415 Are you dealing with heavily partitioned data? (Those seem to be the people happiest with Tables.) |
| mh415: | @smarx: yep. i'm all about partitions. |
| pstatho: | partitions rock! |
| argodev: | considering where I work, and what I'm working on, I'd love to have an offline conversation about Azure and HPC |
| smarx: | Yay partitions! :) |
| Gaurav: | With table storage, what are your plans for improved query support. I believe this is most common problems with Tables are this time. |
| smarx: | @argodev Email me any time. Steve.Marx@microsoft.com |
| pstatho: | Now that it's so easy to partition, I'm storing so much more data, things I would not have normally done with RDMS |
| smarx: | @Gaurav What more would you like to see? Secondary indexes are coming in the future (to enable things like querying on non-key properties and sort). |
| mh415: | As a whole, I'm very impressed with Azure, and .NET for that matter. I never owned a Windows machine until I started hearing about it after PDC '08. It's still something of a side interest for me, but I'd like to start using it full time. |
| smarx: | @Gaurav I believe simple aggregation (count, sum) are coming down the road also (but don't hold me to that). |
| smarx: | Any ideas for (small) demos that would be useful? |
| argodev: | re: partitions; I'm currently half-way through loading 350,000,000 entities spread across 41,000 partitions and the insert time has been near linear. |
| smarx: | I like to occasionally write some code... wondering if there are some interesting demos that would help people. |
| smarx: | @argodev That's great. |
| smarx: | Hey, Roger. How's it going? |
| Gaurav: | Another question on table storage. Are you planning to provide some kind of security model (A.K.A ACL) like you did for Containers & Blobs. |
| smarx: | @Gaurav I've been meaning to ask Brad (our storage architect) that question. Not sure. |
| pstatho: | @argodev so that's about 8500 entities per partition, have you tries querying that? I seem to have issues loading data when there are >1000 entities i.e. pretty slow |
| mh415: | RogerJ: I ordered your book yesterday. Looking forward to its arrival. |
| RogerJ: | About the same. Trying to finish the book. |
| RogerJ: | @mh415: Thanks for ordering the book. It should be printed in early October. |
| argodev: | @pstatho not too bad, but in my use case, I'm simply pulling down the entire partition set (i.e. give me all of the entities in in the partition). I've not been sub-dividing further |
| smarx: | @pstatho Well, you can only get up to 1000 entities in a single query, so more than that means multiple roundtrips. |
| pstatho: | yeah even factoring round trips it is taking too long |
| pstatho: | So I start my own threads to all the data I need to put in memory (cache) since there are no secondary indexes |
| RogerJ: | @smarx: Mary Jo Foley says HealthVault runs on Windows Azure. Can or someone else at MSFT confirm this? |
| smarx: | Well, our storage guys always love investigating a good performance mystery. :) So if it's not scaling linearly, you might want to fire off a question on the MSDN forum. |
| Bill___: | Any special Azure parties at PDC :-) |
| mh415: | @smarx: Re: an Azure demo. For me, coming up with a project idea is harder than actually coding it. |
| smarx: | @RogerJ I can't confirm it, no... but follow up with PR: http://www.microsoft.com/presspass/PR_Contacts.mspx |
| smarx: | @mh415 Me too. :) |
| mknz: | @pstatho Rob Gillen did a post demonstrating the seriously negative impact of pulling ATOM as opposed to JSON across the internet from an Azure service - http://weblogs.asp.net/rgillen/archive/2009/08/20/atompub-json-azure-and-large-datasets-part-2.aspx |
| smarx: | @Bill I think we will have a party at PDC, but I don't know if it's all been finalized yet. |
| smarx: | @mknz Note that his experiment was over the internet... a lot less bandwidth there than between nodes in the data center (like when accessing table storage). |
| smarx: | But I'm in total agreement that JSON is a more efficient format. :) |
| mknz: | @RogerJ in a SQL Azure forum thread this morning Patric McElroy refers to "A large (100s of TBs) internal service property has been working with the SQL Azure DB platform for over a year and this past summer went live with the service." They seemingly have had really good performance results - http://social.msdn.microsoft.com/Forums/en-US/ssdsgetstarted/thread/cf5dd5dc-1b55-40f6-97cc-ac13d6ad201f |
| pstatho: | btw, I'm going on a business trip tomorrow, and will be away from the Internet most of the time, but I LOVE the fact I can continue cloud development regardless, props for that |
| mknz: | @smarx I was aware that it was over the internet with the obvious bandwidth issue :) |
| smarx: | @mknz Yeah, just making sure others understood... I heard a lot of people misunderstanding his experiment. :) |
| argodev: | btw, rob gillen == @argodev |
| smarx: | @pstatho Awesome. I'm heading out of town too and plan to write some code on the plane. Woohoo! |
| argodev: | and yes, part 2 hopefully clarified the misconceptions put out in my first post. |
| D__: | waky, it thinks i'm someone else |
| mh415: | For an Azure app to receive email, should I set up my own SMTP server outside of Azure? |
| smarx: | @argodev lol, didn't realize you're the same person. :) |
| smarx: | @mh415 Yeah, or reuse somebody else's. (I use Hotmail, of course, but I hear there's one called GMail too...) |
| mh415: | ;) |
| Gaurav: | one request for #Azure team. Can the future chats be held a little earlier than 12:00 PST. It's 1:30 in the morning in India. I will hang around as long as I can :) |
| smarx: | @Gaurav Yeah, I need to find another time that's more convenient for the other half of the world. :) |
| Gaurav: | @smarx .. thanks |
| argodev: | re: ideas... rumor was someone was working on a map-reduce implementation sample running on azure... that would be interesting |
| argodev: | well... at least to me. |
| pstatho: | to me too |
| mh415: | argodev: I went to a MSR talk about that. No mention of their plans though. |
| smarx: | I built my own MapReduce running on Windows Azure. (It worked pretty much like "Hadoop streaming".) |
| smarx: | The worker + queue model in Windows Azure helps manage this quite nicely. |
| smarx: | But I never really finished up the project. |
| smarx: | Last I left it I was trying to decide between blobs and tables for storage. (Tables seem more natural, but for lots of small pieces of information the Atom overhead was significant.) |
| pstatho: | post it up on code.msdn.microsoft.com so people can take a look and maybe continue the work |
| smarx: | Wade Wegner is also said to have put together some code. |
| argodev: | or codpex |
| smarx: | We've been meaning to huddle together and swap notes. |
| smarx: | Yeah, I did just ask for demo ideas... maybe I should dust off that code and put it out there for you guys to play with. |
| smarx: | Okay, now I know what code to work on on the plane. :) |
| argodev: | re: node-local scratch storage |
| argodev: | docs indicate that you have a quota-restricted space to which you can write/read |
| argodev: | but no indication of what that quota is... |
| az_: | hey .. @smarx , just out of curiosity if you can share, how are the queues implemented in windows azure? database? azure tables? disk? |
| smarx: | @argodev You ask for the space up-front (in .csdef), so you get to pick how much you get. That being said, I think we artificially limit how much you can request quite low (like 20GB IIRC?). That should grow significantly. |
| smarx: | @az_ Under the hood, they're in Windows Azure tables. |
| Gaurav: | and the rationale behind 32 messages? Why 32? |
| smarx: | @Gaurav for how many messages you can retrieve in one API call? |
| smarx: | @Gaurav Well, everything in computers has to be a power of two, right? :) I'm not really sure, I assumed it was a performance tuning thing. I don't know if it will change. |
| Gaurav: | right ... I can only get up to 32 messages per call. |
| Gaurav: | Just curious ... that's all:) Thanks |
| kmarshall: | do you have any recommendations for caching in Azure? I'm working on a pubsubhubbub server in azure. on app engine they have memcache. is velocity coming to azure soon? or is there a good way to use asp.net caching across multiple web instances? |
| az_: | you might have mentioned before, but where can we get this chat transcript from?:) |
| smarx: | @az_ I'm (hopefully) recording it with a mIRC instance on another computer... if that works correctly, I'll post it to my blog (http://blog.smarx.com) soon. |
| smarx: | @kmarshall By PDC, we'll have inter-role-instance communication. So you could run memcached if you wanted. |
| smarx: | @kmarshall There will be one added layer of indirection (an API call to get the IP addresses of the other nodes), but after that you should be able to do what you want. |
| smarx: | @kmarshall Others have been using the .NET Service Bus to publish and subscribe to events. I remember seeing an internal app that used that for cache invalidation. |
| smarx: | A coworker of mine recently had an interesting idea for an application, but one which I couldn't find a reasonable use for... |
| kmarshall: | thats awesome. I was hoping for that. i was going to work something out with the .net service bus, but that's an extra layer i din't want to build if velocity or inter role communication was coming soon |
| smarx: | The app is: you go somewhere and record some information (maybe call a phone number), and then you give someone a pointer to that information. They can access it whenever they want, but as soon as they do, you get notified. |
| pstatho: | I follow azure quite closely and this chat is the first time I'm hearing about inter-role-instance communication, this is the kind of thing that would be nice to communicate to the community |
| pstatho: | poor @kmarshall was about to do a work around |
| smarx: | @pstatho We talked about it at last PDC and at MIX... (basically only at conferences). |
| pstatho: | hmmm, I watched most videos |
| smarx: | So what's a good use case for such an app? It's sort of "information escrow." |
| smarx: | It reminds me a bit of those automated minibars at hotels... you can grab whatever you want, but they'll know and charge you. |
| smarx: | Is there a common consumer use case for the same sort of functionality? |
| pstatho: | r u talking about inter-role communication? |
| Kevin___: | @smarx One that comes to mind is geo-caching. You could get notified if someone found the cache that you hid. |
| smarx: | I'm talking about two things at once. :) inter-role communication we discussed at a few conferences. The "common consumer use case" is about my coworkers app idea. :) |
| smarx: | @Kevin I like that. |
| smarx: | Hmm, the 1:17pm crowd has arrived...? (Three people all at once? :-P) |
| drub0y: | your tweet was affective :) |
| drub0y: | if only somewhat |
| smarx: | But my tweet was a while ago... unless Twitter ate it for a bit. :) |
| drub0y: | err "effective" |
| drub0y: | you're right, 19mins old... i was out getting a Jamba juice :P |
| drub0y: | so i gotta ask, what's up with the 10GB limit on SQL Azure? |
| smarx: | No one from the SQL Azure team here to give a proper answer... so I won't try. |
| smarx: | lol |
| drub0y: | heheh |
| drub0y: | alright |
| smarx: | That's why everyone got quiet in here. |
| smarx: | I don't actually answer anyone's questions. |
| smarx: | BTW, how many were at the chat last month (using Tinychat)? |
| smarx: | Is this (IRC) better? |
| az_: | I like this one .. works fine ... |
| pstatho: | about the same for me |
| argodev: | wasn't here last time, but this seems to be working smoothly. |
| smarx: | The thing I like about this one is you can resize. |
| mknz: | @drub0y Patric McElroy of MSFT explained the 10GB limit as "One of the primary drivers for the DB size limits is customer feedback" Early adopters are expected to be web and departmental apps. http://social.msdn.microsoft.com/Forums/en-US/ssdsgetstarted/thread/cf5dd5dc-1b55-40f6-97cc-ac13d6ad201f |
| smarx: | Tinychat had a pretty small window, so stuff kept scrolling off. |
| pstatho: | that's true |
| drub0y: | how about caching, what's the word with getting velocity clusters in Azure? |
| Gaurav: | any udates on azure world tour esp. in India? |
| smarx: | @drub0y That's what you get for going to Jamba Juice. We just discussed that. :) |
| Gaurav: | how would one sign up? |
| drub0y: | damn it! |
| smarx: | @drub0y By PDC we'll have inter-role-instance communication, so you can run memcached or whatever. |
| smarx: | There will be a layer of indirection though (call an API to get the IP addresses of the other nodes). |
| drub0y: | sure |
| pstatho: | how will upgrades be communicated, ex when .NET 4 is released how long after can we expect it on Azure? |
| drub0y: | just thinking about how that would help me with velocity right now |
| smarx: | @Gaurav I don't know if there's sign-up yet for India. It looks like, BTW, I'll make it to London, Munich, and Paris. (Not India, unfortunately.) |
| Gaurav: | (; |
| smarx: | @pstatho We plan on grabbing .NET 4.0 once it RTMs. In general, the model is still being worked out for how exactly you maintain control over what updates you get and when. |
| pstatho: | I take it it will be the same for PHP releases |
| drub0y: | mknz, thanks for that forum link... good info |
| smarx: | For PHP you actually package the whole runtime together with your app. |
| smarx: | So you can put whatever version in there you want. |
| smarx: | (But it does mean it's on you to grab the new version and redeploy when some security patch comes out.) |
| pstatho: | ok but what about the accelarator that was recentely beta released? |
| argodev: | this is touching back to what was sort of asked earlier, but is there any notion of adding block-level access b/t worker roles and blob storage? |
| pstatho: | http://blogs.iis.net/ruslany/archive/2009/09/01/windows-cache-extension-1-0-for-php-beta.aspx |
| smarx: | @argodev Nothing to talk about now. |
| IndyBob: | @smarx: Is there a primary cost before usage expense or a minimum monthly fee before usage calculation? |
| smarx: | @pstatho Um, good question. Is it something that has to be installed as admin? |
| kmarshall: | are there any plans to add a web interface to the storage in your azure account? Like viewing logs online, or just querying records to view, delete, etc? |
| smarx: | @IndyBob The pricing we announced is pure consumption, but there will also be other models (e.g. subscription) that are a bit different. But for the consumption model, there's no cost except what you consume. |
| drub0y: | are you guys working on providing native language (.NET, Java, PHP, etc.) client libraries for the various core azure APIs? the samples are "ok", but they're not great and I would think you'd see faster adoption if people could just reference a pre-baked client library and go to work |
| IndyBob: | Thanks |
| az_: | @smarx ... any internal microsoft products moved fully to windows azure? .. or in the process of |
| smarx: | @kmarshall I've seen some prototypes float around... not sure there will be an "official" one, but there are plenty of projects that do something like this. |
| pstatho: | afaik, IIS extensions require admin rights |
| smarx: | @drub0y We are working on libraries for a few languages (have you seen the PHP storage SDK?). |
| smarx: | @pstatho Hmm, then we'll have to consider taking it ourselves (probably not until it RTMs). |
| smarx: | @az_ The short answer is "yes" (some that are fully moved and running on Windows Azure now, and many others that are in the process of moving). I don't know which ones I'm allowed to talk about though. :) |
| gtsiokos: | What about running SMTP servers in Azure? If not, will there be a Azure SMTP service for receiving incomming mail? |
| Kevin___: | @kmarshall here is a codeplex project showing a web interface to Windows storage http://azurestorageexplorer.codeplex.com/ (disclaimer: haven't actually tried this out, but the pics look good) |
| kmarshall: | yeah, I built my own to do crud operations on the tables, but the log viewing is just a pain. Having to click copy and wait for them to get transferred over to storage and then download to view. It would be nice to just view the logs in the azure portal |
| RogerJ: | @az_ Mary Jo Foley has some information on MSFT products running on Azure. |
| smarx: | @gtsiokos It should be possible to run an SMTP server in Windows Azure by commercial launch. But I would warn against relying on that... there's a decent chance someone will use Windows Azure to send a bunch of spam and get our whole IP range blacklisted. |
| Gaurav: | @smarx .. is it ok to pitch for my product here :) |
| smarx: | @Gaurav By all means, please do. :) |
| az_: | RogerJ .. I'll check it out .. thanks .. |
| smarx: | @gtsiokos So my best advice is to use an external SMTP server like Hotmail or GMail, or your own that runs outside the cloud. |
| Gaurav: | tools ... have you guys seen Cloud Storage Studio? |
| mh415: | Gaurav: never heard of it |
| argodev: | @kmarshall yes... the log mechanism in azure is painful for figuring out what is going wrong |
| smarx: | FYI, the way logs work is also evolving (into something more pluggable and easier to use live). |
| Gaurav: | I believe some here are already using it .. send me an email gmantri@cerebrata.com or visit our website www.cerebrata.com. |
| argodev: | and, unless I'm mis-interpreting what is happening, if your node goes down (i.e. exception/etc/restart is called) your logs go to the bit bucket in the sky |
| pstatho: | Cloud Storage Studio is the best GUI azure management software that I've used |
| argodev: | I've been using a queue dedicated to status tracking and pulling from there for more immediate feedback when needed. |
| Gaurav: | @pstatho ... thanks |
| smarx: | There's a nice endorsement, Gaurav :) |
| kmarshall: | gaurav. that's cool and i had a nice SL one built, but I don't think I want to put my credentials for my production azure account into someone's SL app and all my data has to go through them. |
| mh415: | Re: Cloud Storatge Studio. I can't say I'm comfortable shaing my account credentials with 3rd parties. |
| Gaurav: | i am still waiting to hear from you :) |
| Azteca: | @smarx. Fries with that? And want lettuce, tomato, onion on the side? |
| Gaurav: | @mh415 .. we have a WPF app as well. You don't have to share your credentials there. |
| pstatho: | @guarav I know, I'll reply this weekend |
| smarx: | @Azteca Yes. |
| mh415: | Gaurav: ah, much better. |
| pstatho: | So @smarx you mentioned we'll be able to run an SMTP server at launch (I'm not interested in running an SMTP server), I am interested in knowing how that works, we will have admin rights on the server? RDP? |
| smarx: | @pstatho No, but you'll be able to receive traffic that doesn't come through IIS. |
| pstatho: | ok interesting |
| smarx: | @pstatho So I'm not sure which SMTP servers can run this way (non-admin, no install), but I would imagine there are some reasonable solutions. |
| smarx: | BTW, don't hold me to this... because I'm not 100% sure the communication we'll allow covers everything you'd need for an SMTP server. |
| pstatho: | So I can create my own service, that listens on a TCP port and it will be accessible? |
| smarx: | @pstatho I believe so. |
| pstatho: | wow good to know! |
| pstatho: | no I won't hold you to it |
| smarx: | @pstatho :) |
| smarx: | @pstatho, looking at that PHP cache extension, I actually think you could use it today. |
| az_: | ca I run IE in windows azure? :D |
| smarx: | http://learn.iis.net/page.aspx/678/using-windows-cache-extension-for-php/ suggests you just need to modify php.ini and drop in a dll. That should work with our model for PHP. |
| smarx: | @az_ I doubt it. (No GUI context.) |
| smarx: | @az_ Why would you want to? (Want to render web pages and take screenshots?) |
| pstatho: | nice, I'll take a look at that |
| smarx: | @pstatho Let me know if it works... I hadn't heard of it until you mentioned it to me here. :) |
| pstatho: | btw, just so that there is no confusion, I don't like PHP ;) |
| pstatho: | yeah it's just came out this week |
| smarx: | Hey, what's wrong with PHP? |
| az_: | @smarx .. something like that maybe ... |
| az_: | just curious |
| pstatho: | it's slow and it doesn't compile |
| smarx: | @az_ Yeah, I think you probably won't be able to, but I haven't tried it. |
| pstatho: | i.e no type checking |
| mh415: | PHP just isn't fun. |
| mh415: | The frameworks are weak too. |
| smarx: | BTW, I'm not in disagreement (I don't do much PHP coding), just wondering why people don't like it. |
| Gaurav: | Will there be any RDP capability later on to our VMs Azure? |
| smarx: | @Gaurav I'm not actually sure. |
| smarx: | Anybody do any Erlang coding? |
| pstatho: | but it's fast for making quick changes in the UI |
| smarx: | I've just been starting to learn. |
| mh415: | no Erlang, but I know Scala and have experience with Actors. |
| pstatho: | I'm in charge of several large PHP sites (millions of users/day), and I'm considering moving some to Azure |
| smarx: | Actors ("processes" in Erlang) seem really appealing to me for high-concurrency apps. (Not just performance-wise; the model seems cleaner.) |
| mh415: | indeed |
| smarx: | @pstatho Cool. So a guy who doesn't like PHP is in charge of several large PHP sites? :) |
| az_: | @smarx .. Axum is a much better language than Erlang ;) |
| smarx: | Axum! I never even heard of that. |
| mh415: | it's a MSFT language ;) |
| smarx: | haha |
| az_: | @smarx : http://msdn.microsoft.com/en-us/devlabs/dd795202.aspx |
| az_: | :) |
| az_: | and it work in windows azure as well .. |
| smarx: | Don't tell anyone I hadn't heard of it. :) |
| Gaurav: | another question (just out of curiosity) ... how will your differentiate between genuine heavy load on my site v/s DOS attack? |
| smarx: | Axum sounds very interesting... that's one to add to the list... |
| smarx: | @Gaurav I don't believe we will. |
| smarx: | @Gaurav _Is_ there a difference (outside of your app logic) between the two? |
| pstatho: | @az_ what do you like about Axum compared to Erlang? |
| Gaurav: | In one case I want to pay (genuine heavy load) and another I want to avoid :) |
| kevindente: | So do we just hop in here with questions? :) |
| smarx: | @kevindente Please do. |
| kevindente: | I'm currently running a small site on Azure |
| az_: | @pstatho - I thnk Erlang is more mature, axum is an incubation project |
| smarx: | @Gaurav Awesome answer. :) |
| smarx: | @kevindente What's the site? Can we see? :) |
| kevindente: | As we prepare to transition to RTM, is there way to know how much it would cost me to keep it on Azure, when the meter starts running? |
| kevindente: | http://www.review2q.com/ |
| smarx: | @kevindente Oh, right, I've seen that. |
| az_: | @smarx, aoround how many apps are on cloudapp.net? |
| smarx: | @kevindente Hmm, there's a billing CTP happening right now, but I have zero visibility into it, so not sure how to get you in. Send me an email and maybe I can help? (Steve.Marx@microsoft.com) |
| kevindente: | will do, thanks. |
| smarx: | @az_http://www.bing.com/search?q=site:cloudapp.net :) |
| smarx: | @az_ We're not giving out that specific number, but it's a lot. |
| az_: | so around 8k then ... kwel ! |
| smarx: | @az_ And less than twenty are mine. :) |
| smarx: | @chrishayuk I saw your tweet. Don't use this chat as an excuse to not work. :) |
| pstatho: | @az_ the results show ~30K for me |
| az_: | @pstatho 30k -35%dupes .. just a guesstimate :) |
| smarx: | Yeah, the search results are presumably counting every page found on every app (like every one of my blog posts maybe?), so a highly inaccurate way to count. |
| smarx: | Ok, so we talked about Erlang... how many Python devs here? |
| smarx: | I've been in love with Python for years... but I have yet to play with Twisted. Wondering if anyone here has. |
| smarx: | (And to stay somewhat on-topic, is anyone doing Python in Windows Azure? Would you want to?) |
| pstatho: | I used to do python, but it suffers from perf issues |
| smarx: | @pstatho IronPython is supposedly faster... ever test its perf? |
| mh415: | It's faster than Ruby though. |
| pstatho: | yup |
| pstatho: | for certain things it was faster for others (notably JSON) it wasn't |
| nbest: | how do people in the US find the speed of the Azure service during the day on us time? its great in the morning in the UK, but once the US wakes up I find I can't start / stop / upload & provision new projects |
| pstatho: | but it is a pretty nice language |
| pstatho: | @nbest r u talking about the portal only? |
| drub0y: | i got my Azure accounts setup right after the PDC, if I want to be able to pick a data center, do i need to delete those and recreate or something? |
| drub0y: | because it doesn't look like you can edit once you're locked into a specific DC |
| smarx: | @nbest I see a lot of variability in deploy/run/suspend time... haven't noticed a time-of-day pattern, but it's interesting that you do. (Maybe it really is all the yanks waking up.) |
| pstatho: | I find the portal pretty slow in general (from Canada) |
| smarx: | @drub0y Yeah, there's no way to change regions after you're already running. |
| chrishayuk: | Thanks for calling me out Steve, I'll be back on the book later tonight |
| pstatho: | but my site runs fairly consistent |
| nbest: | yeah, if i'm trying to update a project its nearly impossible at night here in the uk |
| smarx: | @chrishayuk See, not only did you use this chat as an excuse, but then you weren't even paying attention to it for the past 15 minutes. :) |
| chrishayuk: | :) yeah, I got distracted listening to a givecamp interview |
| drub0y: | smarx, so if i delete my services will i be able to reclaim those tokens and recreate or am I going to need to aquire new tokens? |
| smarx: | @drub0y You reclaim the slot. So you can delete and then create a new project. |
| nbest: | also is the time it takes to provision and initialize a project through the Azure portal dependant on the size of the project you are uploading? |
| chrishayuk: | Steve will there surprise features on RTM or will all new features be part of CTP's on the run up, or can you not say? |
| smarx: | @nbest The provision time (from pressing "deploy" to getting to the "allocated" step) is somewhat dependent on package size because there's a copy involved. But running (initializing->started) shouldn't depend on the size. |
| smarx: | @chrishayuk There will be a few things that come right at PDC (as opposed to in an earlier CTP). |
| smarx: | @chrishayuk But I also expect us to have at least one more CTP before PDC. |
| pstatho: | @smarx you mentioned the logging system will change can you give us an insite? will we be able to see logs in (near) real-time like in the dev fabric? |
| smarx: | (But that's always subject to change, yadda yadda yadda.) |
| smarx: | @pstatho Yes, I believe so (but until it's shipped, it's subject to change, yadda yadda yadda). |
| pstatho: | lol |
| chrishayuk: | is there any chance we can have an offline discussion regarding as I kinda like to cover it in the book (will obviously keep stum) until after PDC but will means I can get the book out as early as poss |
| smarx: | @chrishayuk Drop me an email (Steve.Marx@microsoft.com) |
| chrishayuk: | thanks steve, you're a superstar |
| smarx: | Can I quote you on that? :) |
| chrishayuk: | sure can |
| Noopman: | Hello peops! |
| pstatho: | hey |
| nbest: | hi |
| smarx: | Hey, what's up? |
| Noopman: | It's very late here in Sweden and I gotta get up in the morning. I just saw @smarx tweet and thought I'd stop by with two links: http://cloudstorageapi.codeplex.com/ and http://cloudstorageapi.codeplex.com/ |
| Noopman: | We've been doing work on Azure testability, persistance ignorance and extensibility. |
| Noopman: | I have the last bits of code on my machine (The extensibility part.) |
| pstatho: | @noopman both links are the same |
| Noopman: | haha... like I said it's late w8 |
| Noopman: | http://azurecontrib.codeplex.com/ |
| Noopman: | There you go! |
| smarx: | @Noopman is the idea to support more clouds than just Windows Azure? |
| Noopman: | If any one feel like it go ahead and comment on the site or get back to me (noopman @ codeplex or http://blog.noop.se) |
| Noopman: | It sure could be the idea |
| Noopman: | Potentially |
| smarx: | Interesting. |
| Noopman: | Not that we've had the time or anything to look into the (small?) differences between cloud storages |
| Noopman: | As a dev I'd like to not care at all where I store my stuff. One cloud storage is the same as the next for me. |
| smarx: | Well, that's why I ask. Some functionality seems very similar between, e.g. Windows Azure blobs and Amazon S3. |
| smarx: | But others seem quite different. |
| Noopman: | And to be honest I think an initiative like this could bring more devs TO Azure rather than away. |
| smarx: | So I'm always interested when "common API" projects come up like this to see what works out well. |
| Gaurav: | It's getting really late for me 3:00 in the morning ... so thanks everybody ... I really enjoyed it. Thanks @steve. Next time please have it little earlier so that I can stay till the end. |
| smarx: | Sure. |
| Gaurav: | Have a nice memorial day weekend. |
| Gaurav: | sorry labor day!!! |
| smarx: | @Gaurav You got it! Thanks for coming by. |
| smarx: | @Gaurav Eh, most Americans don't know the difference anyway. :) |
| pstatho: | nice work @noopman |
| Noopman: | @smarx I have no idea what the merits of our API suggestion would be in the bigger picture. It's just something that helped us along when fiddling with Azure. |
| Noopman: | thanks @pstatho |
| smarx: | @Noopman I'll dig in a little and take a look. I love that you're using MEF, by the way... I haven't gotten to play with it yet. |
| Noopman: | It works WONDERS! |
| Noopman: | Our worker role loop is just BAD ASS I can tell you. |
| smarx: | :) |
| Noopman: | All it does is WorkService.GetWork(); work.DoWork() - for ever |
| Noopman: | the work service imports the work methods through MEF |
| smarx: | Ha, cool. |
| Noopman: | But that bit of code is sitting on this machine than I'm typing from almost ready to committ to AzureContrib |
| smarx: | Well hurry up! |
| smarx: | What are you doing typing to us when you could be checking in code? :) |
| Noopman: | AND we have a OneTimeWorkService solving the issue of queues being creted more than once. |
| Noopman: | ahahah |
| Noopman: | You know what! I'll go hack away... Just because you said it! Gotta tell the wife I'm going back up again. (Laptop in bed.) |
| smarx: | Hmm, how does that work? |
| smarx: | (Now that you want to go back to work, I ask you a question. :-P) |
| Gaurav: | @Noopman I haven't seen your APIs yet ... but one advantage I can see is that I can have your APIs sit closer to storage engine thus reducing the latency issues. |
| Noopman: | cool |
| Noopman: | The OneTimeWork service imports methods to execute through MEF. The methods have metadata so they may be sorted. Also curtesy of MEF. Then the OneTimeWork service is a shared instance. |
| Gaurav: | @argodev ... then we can have convert the data into JSON or GZIP format and push it out .... |
| smarx: | Ah, I see. |
| Noopman: | @smarx So it keeps a flag for when it's run the methods once. |
| Noopman: | But sure... each method is run once on each instance of a web or worker role |
| Noopman: | We haven't worked around that yet. |
| smarx: | Right, I get it now. |
| Noopman: | Not too many lines of code actually. |
| Noopman: | @smarx The extensibility bits are the most interesting. We can run our work meth |
| Noopman: | woops |
| Noopman: | @smarx Say you have worker 1 and worker 2 with task A and task B respectively. |
| Noopman: | @smarx When worker 1 runs out of tasks A it stands around idling while worker 2 works away on his tasks B. |
| Noopman: | @smarx not our extended workers. |
| Noopman: | @smarx They all work on all tasks. worker 1 can work in the order A-B and worker 2 will work in the order B-A. When worker 1 runs out of tasks A he consequently helps worker 2 with the B tasks. |
| smarx: | I see. Yeah, that's mostly how my workers work, but in a dumb, hand-coded way. (while(true) { if (workA) { doit; } if (workB) { doit; } }) |
| Noopman: | MEF solves the dumbness. |
| smarx: | Yup. |
| Noopman: | You know what I got 127 green tests here. The code is not perfect (never is) but I'll go ahead and committ to AzureContrib right now. W8 |
| chrishayuk: | steve, are you gonna be wearing the red shoes again at the pdc? |
| smarx: | That's a good question. I actually saw a coworker wearing the red shoes today. |
| smarx: | I'll probably end up wearing them. |
| smarx: | Unless we come up with some cool new footwear. (Maybe colored... azure?) |
| kmarshall: | This is probably a really dumb question, but it's bothering me. If i'm debugging a azure web role project in VS and I change a line of html or css, can I not just refresh the browser to see the change? |
| mknz: | @smarx get a bunch of Italian national soccer outfits and come as the Azzurri |
| pstatho: | @kmarshall no because you need to build the package |
| pstatho: | but I have a kind of workaround |
| kmarshall: | that is brutal |
| pstatho: | I do not run F5 to test |
| pstatho: | I start the dev fabric and dev storage through command line (bat files) |
| chrishayuk: | you definately need to do something, azure blue shoes would be cool |
| pstatho: | so I VS I just hit F6 to rebuild the package, and RdRoleHost will automatically restart like IIS |
| smarx: | I agree. I should see if I can find some blue shoes. |
| pstatho: | when it relaods the appdomain |
| smarx: | @pstatho How does your bat file work? Do you run cspack? |
| pstatho: | when you really need to debug I attach to the RdRoleHost process |
| smarx: | @pstatho Or do you just csrun and leave the directory structure as-is? |
| kmarshall: | hmm. I ended up just adding a 2nd non-azure web role and putting all the css/js in there when i develop. Then in the azure web role i just link to those files |
| pstatho: | csrun with the defualt directories |
| kmarshall: | so i can change on the fly |
| Noopman: | Well... here it is: http://azurecontrib.codeplex.com/SourceControl/ListDownloadableCommits.aspx |
| Noopman: | The last check in: A preview of AzureContrib V 3. |
| nbest: | just wondering has there been any downtime to the Azure service offically / unoffically? |
| smarx: | @Noopman Nice, real-time development. :) |
| pstatho: | also @kmarshall, you can still access your site thru regular VS WebDev server |
| smarx: | @nbest There have been a few problems, yes. One somewhat major outage (couldn't get to some hosted sites for hours). All other issues were mostly "the portal doesn't work" or "I can't deploy a new app." |
| Noopman: | @smarx ;-) We only have one algoritm for ordering the work. Meaning RIGHT NOW we don't have workers that work on diffrent prios. |
| smarx: | @nbest Whenever something like that happens, we post a sticky note to the Windows Azure forum on MSDN (http://tinyurl.com/wazforum), so that's a good way to track how well we're doing. |
| pstatho: | @noopman downloading it now , and will look at it this weekend |
| nbest: | @smarx, thanks, i guess there is no offical log of this - i personally haven't experianced this, its just my boss is urging us to release before pdc and i'm slightly concerned about uptime |
| smarx: | I think these are only for kids, but these are pretty sweet azure shoes... http://www.planet-sports.com/en/product43391/manufacturer71/product_info.html |
| pstatho: | @nbest usually it's the other way around ;) |
| Noopman: | @pstatho Lovely. If you want to get back to me with a comment I'd be thrilled! just hook up through codeplex - I'm the coordinator for both projects. |
| smarx: | @nbest We're doing pretty well, but the SLA kicks in at PDC, so if you need guarantees, I'd advise you to wait until then. |
| pstatho: | @smarx seems to be the right hue of blue |
| kmarshall: | I like azure a lot and for larger code bases I'd prefer c# and having the worker roles, but for smaller projects app engine is just way faster to build anything |
| smarx: | @kmarshall Faster just because the deployment time is so fast? Or faster to write the code too? |
| kmarshall: | like the time to deploy, viewing logs - that stuff just kills me. |
| kmarshall: | the code also |
| kmarshall: | i mean the data storage apis are a little simple i think |
| kmarshall: | and in general python is going to be less code |
| kmarshall: | err the storage api on google data are simpler |
| smarx: | Good feedback. |
| kmarshall: | like in app engine i can view any data or logs deployed instantly |
| kmarshall: | i mess one thing up in azure deployment and it's like 30-45 mins to fix |
| kmarshall: | plus i don't know how the issue of having to redeploy in your development fabric to change any html/css/js isn't more of an issue for people |
| smarx: | We are working on both the ease of getting to logs and the deployment time, so hopefully we get better in that regard. |
| smarx: | Yeah, I wonder if there's a way we can improve the tooling around making changes to running apps. It makes sense for web roles, but I'm not sure what should happen for a worker. |
| nbest: | cutting down the time for deployment will be nice, even doing one release per day seems to drag on and on waiting |
| smarx: | @nbest I deployed new code for this site (wazl.cloudapp.net) 30 minutes before the chat started, and it made me pretty nervous. :) |
| kmarshall: | does it provision a new vm each time you redeploy? |
| kmarshall: | or does it just copy the new files? |
| smarx: | @kmarshall Yup. |
| smarx: | Provisions a new VM. |
| nbest: | @kmarshall good question |
| kmarshall: | is there no way to just swap in the new code? |
| smarx: | But that doesn't mean it will always work exactly that way. :) |
| kmarshall: | that would be awesome |
| Noopman: | Well - I'm off to bed. Nice chat room and thank you for listening to my 'rant'! ;-) |
| smarx: | @Noopman Thanks for hanging out! Goodnight! |
| kmarshall: | have you tried app engine at all? not trying to make you compare products, but it deploys faster to production than i can build the azure project in VS |
| smarx: | I've used App Engine a fair bit. |
| chrishayuk: | I've haven't got around to trying but I had an idea where you could store your website in blob storage and then have your webrole sync to blob storage, have been meaning to do a POC on it |
| smarx: | @chrishayuk I tried that and ran into permissiosn issue writing to the root of my web role directory. |
| smarx: | @chrishayuk I believe it works as long as you don't write files in the root (only in subdirectories), but I never finished that investigation. |
| chrishayuk: | yeah but you could store the data back into local storage, and then generate the content semi dynamically with an asp-net output cache |
| smarx: | @chrishayuk Ah, I was trying to just overwrite files. You could definitely use a VirtualPathProvider or something similar to get around that (but those methods have limitations). |
| smarx: | Ok, it's officially 3PM PDT... I'm going to have to head out soon. |
| kmarshall: | thanx for hosting the chat |
| smarx: | But thanks everyone for hanging out and chatting. I'd like to do these pretty regularly (sprinkling in some times that are better for Europe and India). |
| chrishayuk: | thanks for your time steve, we all appreciate it |
| nbest: | that time already, thanks for the info another good chat |
| nbest: | @smarx yeah friday night @ 10pm uk time isn't so great (but still worth it) |
| smarx: | Cool, thanks guys! |